refactor: resolve all 44 audit findings across sigint plugin#3
Merged
Conversation
Replace sigint.local.md (YAML) and .sigint.config.json (v1.0) with unified sigint.config.json v2.0 at project root. Per-topic config indexed by slug with defaults/research/topics blocks. Shared Config Resolution Protocol, /sigint:migrate skill, CONTEXT.md per topic. - New: protocols/CONFIG-RESOLUTION.md shared resolution protocol - New: skills/migrate/SKILL.md with dry-run and merge mode - Updated: start, init, issues skills + hooks for v2.0 config - 22 new evals + 1 updated across commands/integration/orchestration - 8 new evals for issues skill via eval-doctor
- New protocols/TREND-INDICATORS.md — shared INC/DEC/CONST definitions extracted from 8 methodology skills (PROMPT-01) - New evals for augment, report, migrate skills (QUAL-01) — 10 test cases covering happy path, error paths, and edge cases - Add audit-results.md documenting all 44 findings
Security (7):
- Harden dependabot workflow with actor guard + SHA pinning (SEC-01)
- Add <untrusted_data> delimiters at all codex review gates (SEC-02)
- Add <user_input> tags + input sanitization in orchestration skills (SEC-03)
- Document threat model in SECURITY.md with in-scope categories (SEC-04/07)
- Guard apply_command in repo-metadata.json (SEC-05)
- Add .env, *.env, *.bak to .gitignore (SEC-06)
Architecture (16):
- Fix tool permissions in 2 agents + 3 commands (ARCH-01/02)
- Standardize {topic_slug} naming across all files (ARCH-03)
- Enforce blackboard null-guard with file fallback (ARCH-04)
- Fix codex:codex-rescue → codex:rescue at 5 locations (ARCH-05)
- Replace wildcard blackboard key with explicit enumeration (ARCH-06)
- Fix hardcoded team name, duplicate derivation, missing allowed-tools,
error handling, return path, heading/numbering (ARCH-07–16)
Prompt/Skill (11):
- Reference shared TREND-INDICATORS protocol from 8 skills (PROMPT-01)
- Add universal confidence scale to all methodology skills (PROMPT-02)
- Fix Mermaid xychart-beta claim, placeholders, output rules (PROMPT-03–07)
- Standardize skill names to slug format (PROMPT-08)
- Fix date placeholder, interview timing, conditional diagrams (PROMPT-09–11)
Code Quality (10):
- Convert output_matches → regex_match in 4 eval files (QUAL-02)
- Strengthen config cascade and conflict detection evals (QUAL-03/04)
- Remove contradictory prDraft, add .bak protection (QUAL-05/06)
- Add polling, chunk limits, dry-run fallback, naming docs (QUAL-07–10)
Scores: Clean Code 8/10, Architecture 8/10, Security 4/10 → 7/10
- Add Agent to skills/report/SKILL.md allowed-tools (needed to spawn report-synthesizer) - Add Bash to agents/research-orchestrator.md tools list (needed for mkdir -p in Phase 0.2)
There was a problem hiding this comment.
Pull request overview
This PR addresses the full set of audit findings for the sigint plugin by tightening security controls, standardizing orchestration conventions (notably topic_slug), extracting shared protocols (config resolution + trend indicators), and expanding/strengthening eval coverage across skills, agents, commands, and documentation.
Changes:
- Standardize orchestration variables and tool permissions across skills/agents/commands; add resilience patterns (timeouts, blackboard/file dual-write, error handling).
- Improve prompt-injection and supply-chain security (input sanitization +
<user_input>/<untrusted_data>boundaries; dependabot actor guard + SHA pinning; broaden SECURITY.md). - Add shared protocols (
CONFIG-RESOLUTION,TREND-INDICATORS) and extend eval coverage (new skill eval suites + regex-based deterministic checks).
Reviewed changes
Copilot reviewed 44 out of 45 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| skills/update/SKILL.md | Adds tool allowlist, input sanitization, <user_input> wrapping, topic_slug standardization, and timeout/partial-results guidance. |
| skills/trend-modeling/SKILL.md | Refactors trend indicator definitions to reference shared protocol; adds universal confidence tiers. |
| skills/trend-modeling/evals/evals.json | Converts checks to regex_match for trend-modeling eval assertions. |
| skills/trend-analysis/SKILL.md | Refactors trend logic section to shared protocol reference; adds universal confidence tiers. |
| skills/tech-assessment/SKILL.md | Adds shared trend indicator protocol reference + universal confidence tiers. |
| skills/start/SKILL.md | Adds tool allowlist, input sanitization, config resolution protocol usage, <user_input> wrapping, and error handling. |
| skills/start/evals/evals.json | Updates expectations from topic-slug to topic_slug naming and corresponding text. |
| skills/report/SKILL.md | Reorders/normalizes allowed-tools, adds sanitization, topic_slug propagation, date placeholder instruction, and <user_input> wrapping. |
| skills/report/evals/evals.json | Adds new unit-level evals for the report orchestration workflow. |
| skills/report-writing/SKILL.md | Fixes Mermaid line-chart guidance to use xychart-beta; adjusts full-report visualization conditions. |
| skills/report-writing/evals/evals.json | Converts legacy array eval format to {skill_name, evals} and updates deterministic check schema. |
| skills/regulatory-review/SKILL.md | Standardizes skill name slug, adds trend indicator protocol ref, updates disclaimer wording, adds universal confidence tiers. |
| skills/migrate/SKILL.md | Introduces new migration skill for legacy → v2 config with dry-run, backups, and topic discovery/context scaffolding. |
| skills/migrate/evals/evals.json | Adds unit-level evals for migrate (happy path, nothing-to-migrate, dry-run). |
| skills/market-sizing/SKILL.md | Standardizes skill name slug, adds trend indicator protocol ref, removes placeholder example value, adds universal confidence tiers. |
| skills/market-sizing/evals/evals.json | Converts legacy output_matches checks to regex_match. |
| skills/issues/SKILL.md | Expands/normalizes allowed-tools, adds sanitization, uses config resolution protocol for repo selection, fixes team name template, wraps user inputs. |
| skills/issues/evals/evals.json | Adds comprehensive unit-level eval suite for issues skill (repo resolution, dry-run, labels, no session). |
| skills/financial-analysis/SKILL.md | Standardizes skill name slug, adds trend indicator protocol ref, removes placeholders in scenario table, adds universal confidence tiers. |
| skills/customer-research/SKILL.md | Standardizes skill name slug, adds trend indicator protocol ref, adds interview applicability note, adds mandatory output rules + validation checklist + confidence tiers. |
| skills/competitive-analysis/SKILL.md | Standardizes skill name slug, adds trend indicator protocol ref, adds universal confidence tiers. |
| skills/augment/SKILL.md | Normalizes allowed-tools ordering/completeness, adds sanitization, improves section labeling, wraps user inputs in <user_input>. |
| skills/augment/evals/evals.json | Adds new unit-level evals for augment (happy path, no session, ambiguity, explicit methodology). |
| SECURITY.md | Adds threat model + in-scope categories + GitHub Security Advisories link; updates supported versions. |
| protocols/TREND-INDICATORS.md | Adds canonical shared definitions for INC/DEC/CONST + notation + usage rules. |
| protocols/CONFIG-RESOLUTION.md | Defines canonical cascade-based config resolution + context file loading behavior. |
| hooks/hooks.json | Updates session-start hook text to reference new config files and migration path. |
| evals/orchestration/evals.json | Updates topic_slug naming + regex_match usage; adds mono-repo multi-topic isolation eval. |
| evals/integration/evals.json | Strengthens conflict-resolution checks; adds multiple integration evals around v2 config resolution behaviors. |
| evals/commands/evals.json | Updates init/config expectations to v2 config and adds migrate/init regression tests. |
| evals/agents/dimension-analyst/evals.json | Converts a remaining output_matches check to regex_match. |
| docs/reference/configuration.md | Rewrites configuration reference for v2 JSON schema, cascade rules, and storage layout. |
| commands/status.md | Adds missing blackboard tool permission and updates topic_slug placeholder in blackboard scope. |
| commands/resume.md | Adds AskUserQuestion permission and updates topic_slug placeholders in file paths. |
| commands/report.md | Adds Mermaid validation tool permission to support report diagram generation. |
| commands/init.md | Replaces legacy sigint.local.md guidance with Config Resolution Protocol and v2 config template + legacy detection. |
| audit-results.md | Adds full audit findings report used as the change driver for the refactor. |
| agents/source-chunker.md | Fixes tool list ordering/completeness; adds explicit chunk size cap and SendMessage-based return path. |
| agents/research-orchestrator.md | Fixes tool permissions, topic_slug standardization, blackboard null-guard + dual-write, codex subagent type, <untrusted_data> boundaries, polling guidance, and ID stability guidance. |
| agents/report-synthesizer.md | Adds missing tool permissions; replaces wildcard blackboard reads with explicit per-dimension reads; standardizes topic_slug placeholders in paths. |
| agents/issue-architect.md | Adds missing tool permissions and adds local JSON fallback when GitHub MCP/gh aren’t available; standardizes topic_slug paths. |
| agents/dimension-analyst.md | Fixes tool permissions; renumbers gating steps; standardizes topic_slug placeholders and dual-write guidance. |
| .gitignore | Switches to gitignore sigint.config.json and adds .env / *.env / *.bak patterns. |
| .github/workflows/dependabot-automerge.yml | Adds dependabot actor guard and pins reusable workflow to a specific SHA. |
| .github/repo-metadata.json | Adds “MANUAL ONLY” warning note for apply command to prevent accidental execution. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Reconcile conditional diagram rules in report-writing (PROMPT-11) - Align docs topic-slug placeholders to topic_slug convention - Remove user_input tags from recall_memories query in report skill - Nest default_repo under defaults block in issues eval fixtures - Tighten TeamCreate regex patterns from OR to sequence matching Resolves review comments on PR #3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Changes
Security (7 findings)
dependabot-automerge.yml— actor guard (dependabot[bot]) + SHA pinning<untrusted_data>XML delimiters at all 4 codex review gates<user_input>tags + input sanitization (200-char truncation, backtick/angle-bracket stripping) in all 5 orchestration skillsapply_command_note: "MANUAL ONLY"guard in repo-metadata.json.env,*.env,*.bakadded to .gitignoreArchitecture (16 findings)
{topic-slug}→{topic_slug}across all source filescodex:codex-rescue→codex:rescuePrompt/Skill Architecture (11 findings)
protocols/TREND-INDICATORS.md— shared INC/DEC/CONST definitions referenced by 8 skillsCode Quality (10 findings)
output_matches→regex_matchacross 4 eval filesQuality Scores
Test Plan
output_matchesremaining (all converted toregex_match){topic-slug}remaining in source filescodex:codex-rescueremaining